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Claim 1 57. (Amended) A system for automatically videoing the activities movements of one or more 
participants or objectsr as they move about within a predefined the entire performance area; 
of a sporting, theatre or concert event, during a predefin e d throughout die entire time 
duration of the event, comprising: 

a first set of two or mor e multiple stationary overhead cameras , not capable of pari, tilt 
or zoom movement, sufficient to form a contiguous field-of-view of said entire 
performance area, for generating a first video stream of images that together form a 
contiguous v i ew of the predefined area continuously throughout the predefined said 
entire time duration ; 

a first algorithm operated on a computer system exclusively and solely responsive to 
the said first stream of video images for continuously and simultaneously analyzing 
the images from e ach first cameras all cameras in said first set of multiple stationary 
cameras in order to detect the presence of any one or more participants er objects each 
participant or object within each and every camenTs view, to determine for each 
detected participant or object its relative c e ntroid two-dimensional locat i on centroid 
coordinates within that view, and to combine use this determined information from 
e ach and e very atl said first set camera cameras into for updating a real-time tracking 
database of at least the ongoing centroid two-dimensional coordinates centroid 
location of each one or more participants or obj e cts participant or object , relative to 
the said entire predefined performance area, where said real-time tracking database 
includes all current and past said centroid locations determined throughout said entire 
time of said event, and 

a second algorithm operated on a computer system exclusively and solely responsive 
to the said real-time tracking database for dynamically automatically and individually 
adjusting the current view of each on e or mor e- cam e ras camera in a second set of 
movable multiple stationary cameras, capable of pan, tilt and zoom movement distinct 
from the said first set of stationary cameras, wherein each movable said second set 
camera is automatically directed without operator intervention to maintain an 
independent view of one or more the participants or objects, where the said second set 
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of movable cameras outputs a second stream of video images for viewing and 
recording , and where the wherein said second stream of video images is not used to 
either determine any participant's or object's oentroid two-dimensional coordinates 
centroid locations or to otherwise update the said real-time tracking database. 

Claim 1 58. (Amended) The system of claim 157 wherein the contiguous *4ew field-of-view formed 
by the said first set of stationary overhead cameras is substantially parallel to the ground 
surface within the predefined said entire performance area. 

Claim 159. (Amended) The system of claim 1 57 for further providing a three dimensional model of 
the activities movements of one or more participants or objects, further comprising: 

a third algorithm operated on a computer system responsive to the said real-time 
tracking database and the said first or and said second streams of video images for 
determining the ongoing relative three-dimensional coordinates of one or more 
specific, non-centroid locations on each participant or object and for updating the said 
real-time tracking database to include the additional three-dimensional coordinates of 
all detected locations matched with the respective participants or objects. 

Claim 1 60. (Amended) The system of claim 159 further comprising: 

markers adhered onto one or more locations on each participant or object to be 
tracked, where the said markers are detectable by the said first or said second set of 
stationary cameras, and where the said first or said third algorithms now determine the 
locations of the adhered said markers and update the said real-time tracking database 
with the related ongoing three-dimensional coordinates of each d e t e cted marker for 
forming the three dimensional model of each participant's activiti e s movements . 

Claim 161. (Amended) The system of claim 160 wherein the said markers are substantially 
transparent to the participants, further comprising: 

said markers adhered onto participants that reflect, retroreflect or fluoresce primarily 
non-visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non-visible energy that is reflected or 
retroreflected off of the said markers, or emitting energy that is fluoresced in the hon- 
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visible spectrum by the said markers and is detectable by the said first or said second 
set of stationary cameras. 

Claim 162. (Amended) The system of claim 161 wherein the participants are additionally identified, 
further comprising: 

at least one uniquely encoded marker adhered onto an upper surface of each 
participant to be identified, and 

a forth algorithm operated on a computer system for locating and recognizing the said 
encoded markers within the said first stream of video images and for updating the said 
real-time tracking database with each participant's identity matched to their 
coordinates. 

Claim 1 63. (Amended) The system of claim 162 wherein the said uniquely encoded markers are 
substantially transparent to the participants, further comprising: 

said uniquely encoded markers that reflect, retroreflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non-visible energy that is reflected or 
retroreflected off of the said encoded markers, or emitting energy that is fluoresced in 
the non- visible spectrum by the said encoded markers and is detectable by the said 
first set of stationary overhead cameras. 

Claim 164. (Amended) The system of claim 157 wherein the participants are additionally identified, 
further comprising: 

at least one uniquely encoded marker adhered onto an upper surface of each 
participant to be identified, and 

a third algorithm operated on a computer system for locating and recognizing the said 
encoded markers within the said first stream of video images and for updating the said 
real-time tracking database with each participant's identity matched to its their 
coordinates. 

Claim 1 65. (Amended) The system of claim 164 wherein the said uniquely encoded markers are 
substantially transparent to the participants, further comprising: 
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said uniquely encoded markers Chat reflect, retroreflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

one or more energy sources emitting non-visible energy that is reflected or 
retroreflected off of the said markers, or emitting energy that is fluoresced in the hon- 
visible spectrum by the said markers and is detectable by the said first set of stationary 
overhead cameras. 

Claim 1 66. (Amended) The system of claim 1 57 for videoing the activities movements of two or 

more participants, wherein the said second set of moveable stationary camera s, capable; of 
pan, tilt and zoom movement, comprises at least two cameras and wherein the said second 
set is additionally directed to automatically reassign any one or more of said second set 
cameras currently following any one or more participants to instead follow a different one 
or more participants based upon which camera views are currently blocked by one or more 
participants in front of another, or upon which camera views are best situated to capture the 
programmatically desired view of any given participant or game object. 

Claim J 67. (Amended) A method for automatically videoing the activities movements of one or 

more participants or objects as they move about within a predefined the entire performance 
area of a sporting, theatre or concert event, durinc Q predefin e d throughout the entire time 
duration of the event , comprising the steps of: 

capturing a first stream of video images using a first set of two or more multiple 
stationary overhead cameras, not capable of pan* tilt or zoom movement, wh e rein the 
combined view from all first cam e ras cov e rs the sufficient to form a contiguous field- 
of-view of said entire predefin e d performance area, and where each said first set 
camera provides images continuously throughout th e predefined said entire time 
duration ; 

simultaneously analyzing only the said first stream of video images coming from eaeh 
said first camera set of multiple stationary overhead cameras in order to continuously 
detect the presence of any one or more participants or objects within each and every 
camera's view, to determine each detected participant's or object's relative centroid 
two-dimensional location centroid coordinates within that view, and to combine use 
this determined information from each and every all said first set cam e ra cameras into 
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for updating a real-time tracking database of at least the ongoing c e ntroid the two- 
dimensional coordinates centroid location of each one or more participants or obj e cts 
participant or object relative to the said entire predefined performance area , where 
said real-time tracking database includes all current and past said centroid locations 
determined throughout said entire time of said event ; 

using the said determined participant and object locations stored in said reaUtime 
tracking database to automatically and individually 4ireet adjust , without the aid of an 
operator, sonrto combination of at loam the pan, tilt or zoom movornontn of each 
camera in a the current view of each camera in a second set of ono or more movabl e 
multiple stationary cameras , capable of pan, tilt and zoom movement, distinct from 
the said first set of stationary overhead cameras, and 

capturing a second stream of video images for viewing and recording using the from 
said second set of automatically movable multiple stationary cameras, wherein the 
said second stream of video images create independent views of one or more of the 
participants or objects within the predefined said entire performance area and where 
the wherein said second stream of video images is not used to either determine any 
participant's or object's centroid two-dimensional coordinates centroid locations or to 
otherwise update the said real-time tracking database. 

Claim 168. (Amended) The method of claim 1 67 wherein the contiguous field-of-view formed by the 
said first set of stationary overhead cameras is arranged so that its combined, contiguous 
v i ew of th e pr e d e fin e d area i s substantially parallel to the ground surface within the said 
entire pr e d e fin e d performance area. 

Claim 169. (Amended) The method of claim 167 for further providing a three dimensional model of 
the acti v i t ies movements of on e or more the participants or objects, comprising the 
additional step of: 

analyzing the said real-time tracking database and both the said first and said second 
streams of video images to determine the ongoing relative three-dimensional 
coordinates of one or more specific, non-centroid locations on each participant or 
object and for updating the said real-time tracking database to include the additional 
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three-dimensional coordinates of all detected locations matched with the respective 
participants or objects . 

Claim 1 70. (Amended) The method of claim 1 69 further comprising the steps of: 

adhering markers onto the one or more locations on each participant or object to be 
tracked; 

detecting the said markers using computer based image analysis of the said first or 
said second streams of video in order to determine the ongoing relative three- 
dimensional coordinates of each marker, and 

updating the said real-time tracking database to indicate the ongoing relative three- 
dimensional coordinates of the said detected markers matched with the respective 
participants or objects. 

Claim 171. (Amended) The method of claim 1 70 wherein the said markers are substantially 
transparent to the participants, further comprising the steps of: 

using said markers that reflect, retroreflect or fluoresce primarily non-visible energy 
and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non-visible energy throughout the said 
predefined entire performance area, to be reflected or retroreflected off of the said 
markers, or emitting energy that is fluoresced in the non-visible spectrum by the said 
markers, where the non-visible energy is detectable by the said first or said second set 
of stationary cameras. 

Claim 172. (Amended) The method of claim 171 for additionally identifying wherein the participants 
are additionally identified , further comprising the steps of: 

placing at least one uniquely encoded marker onto a - top an upper surface of each 
participant to be identified; 

detecting and recognizing each encoded marker using computer based image analysis 
of the said first stream of video images, and 


Page 6 of 8 


PACE 16/18 • RCVD AT 9/9/2008 12:05:05 PM [Eastern Daylight Time] * SVR:USPTO-EFXRF-6/14 " DNIS: 2738300 « CSID:Fax 


• DURATION (mm-ss): 06-32 


Fm:Smartek, Inc. To:App SN 10/006,444 312 Amendment (15712738300) 


12:05 09/09/08GMT-04 Pg 17-18 


312 Amended Claims 
(with additions and deletions marked) 

updating the said real-time tracking database to indicate the identity of each 
participant matched to their coordinates. 

Claim 1 73. (Amended) The method of claim 1 72 wherein the uniquely encoded markers are 
substantially transparent to the participants, further comprising the steps of: 

using said uniquely encoded markers that reflect, retroreflect or fluoresce primarily 
non-visible energy and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non-visible energy throughout the 
predefined said entire performance area, to be reflected or retroreflected off of the said 
encoded markers, or emitting energy that is fluoresced in the non-visible spectrum by 
the said encoded markers, where the non-visible energy is detectable by the said first 
set of stationary overhead cameras. 

Claim 174. (Amended) The method of claim 1 67 for additionally identifying wherein the participants 
are additionally identified , further comprising the steps of: 

placing at least one uniquely encoded marker onto a top an up per surface of each 
participant to be identified; 

detecting and recognizing each said encoded marker using computer based image 
analysis of the said first stream of video images , and 

updating the said real-time tracking database to indicate the identity of each 
participant matched to their coordinates. 

Claim 175. (Amended) The method of claim 174 wherein the said uniquely encoded markers are 
substantially transparent to the participants, further comprising the steps of: 

using uniquely encoded markers that reflect, retroreflect or fluoresce primarily non- 
visible energy and are therefore substantially visibly transparent, and 

using one or more energy sources to emit non-visible energy throughout the 
pr e def i ned said entire performance area, to be reflected or retroreflected off of the said 
encoded markers, or emitting energy that is fluoresced in the non-visible spectrum by 
the said encoded markers, where the non-visible energy is detectable by the said, first 
set of stationary overhead cameras. 
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Claim 1 76. (Amended) The method of claim 1 67 for videoing the activities movements of two or 
more participants from optimal viewpoints, further comprising the steps of: 

using two or more cameras in the said second set of movable stationary cameras, and 

during the step of individually directing at least adjusting the pan, tilt or zoom 
movements of each camera in the said second set of stationary camera s, capable of 
pan, tilt and zoom movement dynamically considering the location of each 
participant or object with respect to the view of each said second set camera and 
automatically reassigning any one or more said second set cameras currently 
following any one or more participants or objects to instead follow a different one or 
more participants or objects based upon which camera views are best situated to 
capture the programmatically desired view of any given participant or game object. 
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